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SPECIFICATION 
PLAYLIST GENERATION METHOD AND APPARATUS 

BACKGROUND 

Field of the Disclosure 

The present disclosure relates to data communications, and, in particular, 
to a novel system and apparatus for the automatic generation of media playlists. 

The Prior Art 

Background 

One application of the Internet that has received considerable attention is 
the ability to transmit or stream media content over the Internet. Once an audio 
or video work has been digitally encoded it may be both downloaded by users 
for play, or broadcast ("streamed") over the Internet. When works are streamed. 
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they may be listened to or viewed by Internet users in a manner much like 
traditional radio and television stations. 

Given the widespread use of digital media, audio works, or video works 
may need to be identified. The need for identification of works may arise in a 
5 variety of situations. For example, an artist may wish to verify royalty payments 
or generate their own Arbitron®-like ratings by identifying how often their 
works are being streamed or downloaded. Thus, playlists of media may need to 
be generated. The prior art has made efforts to create methods for identifying 
g digital works and generating playlists. 

30 As is known by those skilled in the art, a playlist is the documentation of 

2 the performance of one or more works at a particular time over a particular 
media. 

Figure 1 shows a playlist generation system 100 of the prior art. The 
15 system 100 may include one or more remote detection modules 102 deployed at 
various locations throughout a broadcast area. Each detection module 102 may 
include an antenna 104 for receiving broadcast signals and providing the signals 
to an analysis and lookup module 106. The module 106 is typically configured to 
identify the content of the received signal by comparing its audio content against 
20 a database 108 of known reference signatures. 
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If a match is made, typically the module 102 will keep a record of all 
matches made during a predetermined period of time on the database 108. For 
example^ the module 102 may keep a record of song titles detected during a 24- 
hour period. 

5 The system 100 may further include a play list server 110 having a 

processor 112 and database 114. The server 110 is typically configured to receive 
information such as the titles of identified songs from the one or more detection 
modules 102 through a network such as the Internet 109 and generate a playlist 
iQ which may be stored on database 114. 

Jflo The system 100 of the prior art in operation typically requires the identity 

J2 of an imknown work to be determined by the individual detection modules. The 

playlist server 110 is typically only configured to receive and compile identities 
fU of works from the modules 102 into a playlist. 

SUMMARY 

15 A playlist generation system is disclosed. In one aspect, the system may 

include at least one analysis module for receiving and analyzing an iinknown 
work and generating a corresponding representation thereof, and at least one 
identification (ID) server for receiving the representation from the at least one 
analysis modules and determining the identity of the imknown work. 
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The system may receive imknown works from networked sources or 
broadcast sources, or directly from the source prior to the work being provided 
to a network or broadcast source. The analysis modules and ID servers may be 
coupled over a network, such as the Internet. 

5 The representations and identification may be based upon feature vectors, 

a spectral representation of the unknown work, the text output of a speech 
recognition system, musical score produced by a music transcription system, or a 
„ bit calculated key method, such as MD5 hash, as are known in the art. 

14 The system analysis modules may receive a plurality of streaming sources 

M 

CBO for analysis at a single location, or from a plurality of streaming sources for 
analysis at a plurality of access points of the network. 

=:J'5 The system provides representations in approximately real time, and the 

a system may generate a playlist of identified works. The ID server may generate a 
playlist of identified works received from different access points of the network 
15 in approximately real time. 

Various methods for generating a playlist are also disclosed. 



BRIEF DESCRIPTION OF THE DRAWING FIGURES 



Figure 1 is a prior art diagram of a system. 
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Figure 2 is a block diagram of one aspect of a disclosed system including 
one or more analysis modules and ID servers. 

Figure 3 is a block diagram of one aspect of an analysis module. 

Figure 4 is a flowchart of one aspect of a disclosed system. 

Figure 5 is a flowchart of one aspect of a disclosed system. 
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DETAILED DESCRIPTION 

Persons of ordinary skill in the art will realize that the following 
description is illustrative only and not in any way limiting. Other modifications 
5 and improvements will readily suggest themselves to such skilled persons 
having the benefit of this disclosure. 

This disclosure may relate to data communications. Various disclosed 

n aspects may be embodied in various computer and machine readable data 

[J 

^ structures. Furthermore, it is contemplated that data structures embodying the 

jo teachings of the disclosure may be transmitted across computer and machine 

^ readable media, and through communications systems by use of standard 

4 protocols such as those used to enable the Internet and other computer 

3 networking standards. 

The disclosure may relate to machine readable media on which are stored 
15 various aspects of the disclosure. It is contemplated that any media suitable for 
retrieving instructions is within the scope of the present disclosure. By way of 
example, such media may take the form of magnetic, optical, or semiconductor 
media. 
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Various aspects of the disclosure may be described through the use of 
flowcharts. Often, a single instance of an aspect of the present disclosure may be 
shown. As is appreciated by those of ordinary skill in the art, however, the 
protocols, processes, and procedures described herein may be repeated 
5 continuously or as often as necessary to satisfy the needs described herein. 
Accordingly, the representation of various aspects of the present disclosure 
through the use of flowcharts should not be used to limit the scope of the present 
disclosure. 

1 Exemplary structure 

ao Figure 2 is a schematic diagram of one aspect of a disclosed playlist 

2 generation system 200. The system 200 may include at least one analysis module 
202 for receiving an unknown work from a broadcast source 204 as is known in 

U the art. The analysis module may include one or more conventional antennae 

3 214 coupled to the analysis module 202 through an input port 210. The input 
15 port 210 may include a conventional receiver for the reception of desired 

broadcast signals. The input port 210 may also be configured to provide remote 
control functionality for allowing the remote control and configuration of the 
receiver, such as providing for remote tuning. The input port 210 may be further 
configured to digitize received signals in digital formats using protocols known 
20 in the art, such as PCM. 
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The analysis module 202 may also be configured to receive an unknown 
work from one or more networked sources 206. In one aspect of a disclosed 
system, the input port 210 of the analysis module 202 may be configured to 
monitor sources providing content in standard formats such as Real®, 
5 QuickTime®, Windows Media®, MPS®, and similar formats, using hardware 
and software as is known in the art. 

In another aspect of a disclosed system, the input port 210 may be 
configured to directly receive audio or video through any of the various means 
know in the art, such as a microphone or video acquisition system. These 

14 0 unknown works may also be provided in standard formats such as MPS, 
m Windows Media, and similar formats. Thus, the analysis module 202 may be 

configured to receive an unknown work prior to the unknown work being 
U presented to the broadcast system or network source. It is envisioned that this 
C3 presentation could occur almost simultaneously. 

1 5 The input port 210 may be operatively coupled to a network 208 through 

which the source 206 may be accessed. The network 208 may comprise any 
packet- or frame-based network known in the art, such as the Internet. The input 
port 210 may also be configured to access the network 208 through any means 
known in the art, such as through traditional copper connections. Furthermore, 

20 the input port 210 may also be configured to access the network 208 using 
wireless connectivity methods as known in the art, including low-power 
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broadband methods such as Bluetooth®, or cellular-based access methods such 
as those used to provide wireless connectivity to cellular phones and personal 
digital assistants (PDAs). 

The analysis module 202 may also include an output port 212 for 
5 providing connectivity to the network 208. The output port 212 may comprise a 
separate unit within the analysis module 202 and may include hardware and 
software to provide the same functionality as the input port 210. Additionally, it 
is contemplated that the output port 212 may comprise substantially the same 
3= circuitry as the input port 210 in order to save space and cost. 

C'l 0 Referring now to Figure 3, a conceptual block diagram of one aspect of a 

53 disclosed analysis module 202 is shown. The analysis module 202 may include a 

O 

processor 302 for operating various aspects of the present disclosure. The 
?u processor 302 may be operatively disposed between the input port 210 and 
u output port 212. 

15 It is contemplated that any processor known in the art may be employed in 

the module 202, and the choice of a processor may depend on the application. 
For example, if the module 202 is embodied in a personal computer, the 
processor 202 may comprise a microprocessor capable of running conventional 
operating systems such as Microsoft Windows®, while if the module 202 is 

20 deployed in a mobile unit such as a PDA, the processor 202 may need only be 
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capable of running an operating system such as Palm OS®, or other embedded 
systems such as may be present in a cell phone or other consumer device. 

The module 202 may include ancillary hardware and software, such as 
conventional memory 304 and a conventional database 306 for the storage and 
5 retrieval of various aspects of the disclosed system and data. 

The module 202 may be configured to generate a representation of an 
unknown work which may then be used by the system to identify the unknown 
work. It is contemplated that a wide variety of methods may be used by the 

y analysis module 202 to generate the representation. The analysis module may be 

y 

i 0 configured to generate a representation of the imknown work using the 

0 psychoacoustic properties of the audio content of the unknown work. Such 

methods are known in the art. For example, the analysis module may generate 

-J 

j1 feature vectors as disclosed in US Patent #5,918,223 to Blum, et al., which is 

5 assigned to the same assignee of the present disclosure and incorporated by 

1 5 reference as though fully set forth herein. 

Additionally, the module 202 may use audio or video spectral or wavelet 
representation techniques as are known in the art. For example, other 
representation forms may comprise the text output of a speech recognition 
system, text output of a close captioned transmission, or a musical score 
20 produced by a music transcription system. In another embodiment, the 
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representation may comprise a bit calculated key using any of the techniques as 
are known in the art such as MD5 hash and CRC. It is contemplated that a wide 
variety of analysis methods may be employed singly or in combination 
advantageously in the present disclosure. 

Referring back to FIG. 1, the system 200 may further include at least one 
identification (ID) server 220 for identifying an imknown work. The ID server 
220 may identify an unknown work using a representation received from the 
analysis module 202 through network 208. Though FIG. 2 shows the ID server 
220 coupled to the same network 208 as the analysis module 202, it is to be 
understood that the various components of the present disclosure may be 
coupled to different networks at different times. 

The ID server 220 may comprise a computer suitable for running an 
operating system such as Microsoft Windows®, UNIX®, LINUX®, MAC OS®, 
and the like. The ID server 220 may include a conventional processor 222 for 
operation of the server. The ID server may further include associated hardware 
and software known in the art such as a conventional database 224 for storing 
embodiments of the disclosure or data. 

It is contemplated that the ID server 220 may be configured to identify the 
unknown work using a variety of methods known in the art. The method for 
identification may correspond to the method(s) used to generate the 
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representation within the analysis module. For example, the ID server 220 may 
be configured to perform identification using the methods disclosed in US Patent 
#5,918,223 to Blum, et al, if the representation were generated using 
corresponding methods. 

Another example would be the pure spectral representations as are known 
in the art. It is envisioned that other representations such as wavelets may be 
used. The invention could also identify the imknown work from the speech 
recognized text compared against a database of song lyrics using any of a variety 
of methods known to those skilled in the art. 

Yet another example would be any of a number of search techniques as are 
known in the art when the representation is a bit calculated key. 

The system may also identify the unknown work by searching a collection 
of musical works for musical note sequences that correspond to the musical score 
in the representation. 

In another configuration the system may use a combination of 
identification techniques, each of which correspond to a representation of the 
unknown work. By using several identification techniques, the chance of a mis- 
identification may be greatly reduced. 

Though the analysis module and ID server are shown as being located 
separately, it is contemplated that they also may be co-located in a single server. 
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For example, it is contemplated that the analysis module and ID server may each 
be embodied in a single board computer wherein the analysis module and ID 
server are housed in a single unit and operatively coupled through a common 
backplane. 

Exemplary operation 

Figure 4 is a flowchart of one aspect of a disclosed method for 
automatically generating a playlist. The process begins in act 400, where at least 
one unknown is received by an analysis module. The analysis module may 
comprise hardware and software substantially as shown and described above. 

Additionally, one or more of the analysis modules may be configured to 
receive a plurality of streaming sources simultaneously for analysis. It is 
contemplated that the analysis modules may be located and configured to 
receive and analyze a wide variety of content, including analog radio or video, 
digital streaming audio or video, or any other media. 

In act 402, the analysis module then creates a representation of the 
received unknown work as shown and described above. The representation may 
be created by the analysis module by extracting psychoacoustic properties from 
the received unknown work as described above. 

In act 404, the representations created by the one or more analysis modules 
may be provided to an ID server. The ID server may comprise hardware and 
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software as described above. It is contemplated that the ID server may comprise 
a single server, multiple servers networked at a single location, or multiple 
servers located at different locations. 

It is contemplated that the various analysis modules may provide 
5 representations to one or more ID servers in a wide variety of manners. For 
example, all of the analysis modules present in a system may provide 
representations in real-time. Or, different analysis modules may be configured 
to provide representations at different intervals depending on the needs of the 
t end user. The analysis modules may transmit representations every sixty 
So seconds, hourly, or as often as is needed. 

0 In some cases where network connectivity is challenging, the 

representations may be batched up and sent to the ID server(s) once a day or less. 

rj In particularly harsh or secretive conditions, the representations may be stored 
within the analysis modules until the modules could be physically retrieved and 

1 5 operatively coupled to an ID server at another physical location. 

It is contemplated that an out-of-band event may be used to trigger the 
transmission of representations. For example, such a trigger may comprise the 
initialization of a connection to a network, or the activation of media playing 
software or hardware. 



14 



ET161056085US 

AMC-01-005 

Figure 5 is a flowchart of a further disclosed aspect of a disclosed method. 
The process begins with act 500, where an ID server receives at least one 
representation of an unknown work. The representations may be received from 
analysis modules as described above. 

In act 502, the ID server identifies the unknown work based upon the 
representation. This identification may be performed using the methods as 
described above. The identification may include such information as the song 
title, artist, label, or any other information as is known in the art that may be 
associated with the work. The identification information might contain 
information such as the name of the advertiser or a descriptive notation of an 
FCC broadcaster identification segment. 

Once an identification of the unknown work is made, it is contemplated 
that a wide variety of further acts maybe performed. For example, the 
identifications made by the ID server may be used to construct or maintain a 
playlist database. Such a playlist may be stored on the ID server, or on a distant 
server. As will be appreciated by those skilled in the art, if representations are 
provided to the ID server in real-time (or near real-time depending on the 
equipment or network used), a playlist may be generated in corresponding real- 
time. Thus, a playlist may be generated in real-time from inputs provided from 
distant geographic locations or multiple sources that contains a comprehensive 
playlist of every identified media segment. 
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Additionally, the identification may be transmitted back to the analysis 
module which generated the representation. This may be advantageous where it 
is desired to generate a playlist for the particular analysis module's location or 
user. Thus, the ID server may be configured to provide an identification back to 
the source analysis module. 

The identity of the received work may also be used for the maintenance of 
the system. Typically, copies of received works are stored on local drives for 
audit purposes. Since the intermediate representation files may be larger in size 
than the identities, it may be desirable to configure the analysis module to purge 
intermediate files for identified works to recover drive space. It is contemplated 
that the ID server may be configured to transmit the identity of received works 
back to the generating analysis module, and the corresponding representation 
files may then be deleted from local drives by the analysis module, thereby 
recovering valuable capacity. 

Furthermore, it is contemplated that the ID server or analysis module may 
be configured to send information regarding identified works to third parties, 
such as third-party servers. Additionally, the ID server or analysis module may 
be configured to provide an electronic notification to third parties of 
identifications made by the ID server. Examples of electronic notifications may 
include email, HTTP POST transactions, or other electronic commimication as is 
known in the art. As is known by those skilled in the art, these electronic 
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notifications may be used to initiate an action based on their content. For 
example, such notifications may allow the playlist to be accessed in real-time or 
as desired. 

It is contemplated that the ID server may be configured to provide 
5 customized playlists containing information tailored to a customer's individual 
needs. For example, a customer may wish to be notified whenever a certain 
work is broadcast, or whether a particular work is broadcast on a particular 
media outlet. Customers may wish to have complete playlists provided to them 
periodically at desired intervals that may include statistics known in the art. By 
i4 0 using the system as disclosed herein, such requests may be satisfied 

m automatically in real-time, or at whatever interval may be desired. It is to be 

O 

understood that any of the aspects of the present disclosure may be performed in 
21 real time or as often as desired. 

i2 It is contemplated that the ID server may examine the representations of 

15 all segments that were not identified in the process described above, and 

determine that some sub-segments were actually performances of a single work. 
Furthermore, this examination may extract a plurality of other characteristics of 
the original broadcast such as the amount of musical content, amount of speech 
content, a transcription based on speech recognition, the beat of any music 
20 present, etc. These characteristics of the un-identified segments can then be used 
to classify the un-identified unknown representations. For instance, a sub- 
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segment that has been performed many times may be correlated with a high 
amoimt of musical content and a certain minimum length of play time to indicate 
that a new song has been detected. Correlating other values and characteristics 
could indicate that a new advertisement has been detected. In some cases a 
corresponding segment of the original broadcast signal could be retrieved and 
played for a human to perform an identification. 

While embodiments and applications have been shown and described, it 
would be apparent to those skilled in the art that many more modifications and 
improvements than mentioned above are possible without departing from the 
inventive concepts herein. The disclosure, therefore, is not to be restricted except 
in the spirit of the appended claims. 
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